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The MAILING DATE of this communication appears on the cover sheet with the correspondence address 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1 )^ Responsive to communication(s) filed on 22 November 2004 . 
2a)M This action is FINAL. 2b)D This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 1 1 , 453 O.G. 213. 

Disposition of Claims 

4) E3 Claim(s) 1 - 24 is/are pending in the application. 

4a) Of the above claim(s) 14-22 is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1 - 13. 23 - 24 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10)D The drawing(s) filed on is/are: a)D accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1 .85(a). 

Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 
11 )□ The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12)D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)D None of: 

1 .□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 
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1 . The text of those sections of Title 35, U.S. Code not included in this action 
can be found in a prior Office action. 

2. Claims 1 - 4, 6 - 13, 23-24 are rejected under 35 U.S.C. 102(e) as being 
anticipated by Cleron et al ("Cleron"; U.S. Patent Number 6,223,213). 

Regarding independent claim 1 , Cleron discloses a "method of asynchronous 
communication by a plurality of users via a network of interconnected computers" that 
anticipates each and every feature claimed. 

As per "selecting, at a client site, a file and capturing said file as an image file", 
Cleron teaches an e-mail system by which audio and video may be attached to points 
within a created and transmitted document. This can be expressed in HTML, and thus, 
the original "wrapper" in which the audiovisual content arrives is an "image file", (see col 
3, lines 24 - 55). 

The steps of "annotating a portion of the image file with an annotation" and 
"creating, at said client site, an audiovisual message, said audiovisual message relating 
to said file annotation" by "capturing digital information" read upon Cleron's thin client 
that is equipped with a microphone and video input to receive audio and video data (col. 
2 lines 1-3), so as to create and capture digital information for an audiovisual message. 
A step of "appending said image file and said audiovisual message to achieve a 

combined message" occurs in Cleron when the email system allows the thin client to 

capture audio and video data for inclusion with the email message (col. 1 line 66 - col. 

2 line 1). In so doing, the electronic mail being sent out in Cleron is also an audiovisual 

message. 
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The final step of "electronically delivering said combined message to at least one 
of said plurality of users" is part of any e-mail system like Cleron's, where the user can 
click a "Send" link on the rendered email page to send the email message to an 
intended recipient (col. 2 lines 18-20). 

As per claim 2, Cleron, in using HTML pages that contain links to 
audiovisual objects that are attached to them, discloses that "said electronically 
delivering comprises streaming said digital information via said network" (col 7, lines 14 
- 39): rather than sending across the entire video or audio clip, the browser simply 
inserts a token that is representative of the attached clip . The inserted audiovisual 
material is then streamed into the message location "via said network"; the one that 
delivers the attachment. 

As in claim 3's "further comprising creating a reply record, said reply 
record having audiovisual material, said reply record being in response to said 
combined message", Cleron's disclosure of an e-mail environment means that the 
recipient user has the ability to create a reply using the same feature to create an 
audiovisual message to reply in response to the original email. 

As per claim 4, Cleron discloses, in this ability to create an audiovisual 
response, that "said reply record is created by one of said plurality of users, and further 
wherein said one of said plurality of users annotates said image file", since the email 
system allows the thin client to capture audio and video data for inclusion with the email 
message (col. 1 line 66- col. 2 line 3). The overall system is not limited to the creation 
of audiovisual attachments by just the sending user. 



Serial Number: 09/609,609 Page 4 

Art Unit: 2173 

Concerning claim 6's "creating" via "a user interface, said user interface 
having stateful buttons", Cleron's client browser supports a user interface that includes 
pop-up capture panels for both audio and video, with each capture panel enabling the 
user to record a selected clip and add the clip to the email message (col. 2 lines 3-6). 
Such items as a pop-up are typically invoked by "buttons", whereby the system 
undergoes a "stateful" change. 

As per claim 7's "selecting" that "further comprises activating said image 
file for editing" please note that in Cleron the act of attaching audiovisual content to a 
message rendered on the screen from HTML is one of "editing" the "image file" that 
underlies the message. 

Regarding claim 8, Cleron discloses that "said electronically delivering comprises 
transmitting said combined message to a central server for storage and further 
transmission": the host mail server converts the email message and the audio or video 
data to a MIME message (Multipurpose Internet Mail Extensions) and forwards the 
MIME message to the intended recipient where it can be rendered in full (col. 2 lines 
24-27). This use of a server is sufficient to meet the uploading and buffering in the 
claim, since mail servers like Cleron's must hold the information prior to the recipient's 
download. 

Independent claim 9 is similar in scope and substance to claim 1 , and is 
anticipated by Cleron using a similar line of reasoning. 

Concerning claim 10, as discussed above regarding claim 3's "reply record", 
Cleron discloses that "said replying comprises creating an audiovisual reply designated 



Serial Number: 09/609,609 Page 5 

Art Unit: 2173 

for insertion at a predetermined point in said combined message" using an interface like 
that in Cleron's fig. 8, where the replying user can insert the audiovisual annotation at a 
specific location within the overall "combined message". 

As per claim 1 1 , "delivering said audiovisual reply with an address pointer 
indicative of said predetermined point", it has also been noted above that the HTML file 
of a received message has within it addresses to the audiovisual content in the form of 
a token . 

Regarding claim 12, Cleron discloses "storing said combined message at a 
remote location and sending an electronic notification having a uniform resource locator 
corresponding to said location" by having an email system that reads HTML files, which 
are known to access inline insertions like audiovisual annotations by means of a URL. 

Concerning claim 13's "streaming said combined message to said at least one 
desired recipient via the Internet, the MIME-based transmission can be interpreted as a 
form of stream. 

Independent claim 23 is similar in scope and substance to claim 1 , and is 
rejected using Cleron for reasons similar to those given above. As per claim 24's list 
comprising "any of magnetic storage medium, ..." for "the machine readable storage 
medium", the Cleron system is a conventional PC implementation, of the kind that 
relies upon magnetic disk storage. 

3. Claim 5 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Cleron in view of Freivald et al. ("Freivald"; U.S. Patent Number 6,219,818). 
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Cleron shows a browser-based email system that incorporates audio and video 
data with the message document but does not show annotating at least a portion of said 
image file using a cursor-based highlighter. 

But Freivald suggests "annotating at least a portion of said image file using a 
cursor-based highlighter" in that the user can select paragraphs of text bv dragging a 
highlight across the text. Responder 24 then stores the location of the selected text 
and generates one or more CRC for the selected text (col. 7 lines 10-13). This form of 
document region selection for annotative purposes has been used in a wide 
assortment of applications such as Freivald's. 

It would therefore have been obvious to a person of ordinary skill in the art at the 
time of the invention was made to combine Cleron's browser-based email system, 
where an editing process must locate positions for annotative insertions, with Freivald's 
highlighting of a text document. 

The modifications would have been obvious to one of ordinary skill because it 
allows the user to send electronic mails that contain audio, video, and text, to aid the 
user in identify relevant portions of the electronic mail document. 

4. Applicant's arguments filed 22 November 2004 have been fully considered 
but they are not persuasive. 

At page 5 of the remarks, applicant argues that "neither Cleron nor Freivald 
teach, disclose or suggest an 'audiovisual message relating to said file annotation 1 ", nor 
similar limitations found in independent claims 9, 23: "Cleron discloses no more than 
attaching an audiovisual file to an e-mail and Freivald fails to disclose any relationship 
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between highlighting and an audiovisual file". However, the inserted audiovisual 
material in the Cleron e-mail composition arrangement is indeed an "audiovisual 
message", and since it is attached at a particular inline location within an HTML 
message (the "file") and it is therefore an "annotation". Freivald is not relied upon to 
show an audiovisual file, but merely to indicated that it was known in the art to use 
highlighting in the context of marking internet-moderated messages. 

5. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1.136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
extension fee pursuant to 37 CFR 1 .136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

6. Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Raymond J. Bayerl whose telephone number is 
(571 ) 272-4045. The examiner can normally be reached on M - F from 9:00 AM to 4:00 
PM ET. 

7. If attempts to reach the examiner by telephone are unsuccessful, the 
examiner's supervisor, John Cabeca, can be reached on (571) 272-4048. All patent 
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application related correspondence transmitted by FAX must be directed to the central 
FAX number (703) 872-9306. 

8. Any inquiry of a general nature or relating to the status of this application 
or proceeding should be directed to the receptionist whose telephone number is (571) 
272-2100. 




RAYMOND J. BAYERL 
PRIMARY EXAMINER 
ART UNIT 2173 




